Statistical methods for evaluating a crop nitrogen simulation model, N_ABLE

Modelling nitrogen (N) dynamics is a valuable tool to predict the uptake, mobility and leaching of mineral N in soil pro®les. Many crop/soil N simulation models have been developed in the last 20 years for this purpose. However, methods for the operational evaluation of simulation models are not well established. Standard test statistics such as the Fand t-tests are being questioned because they may give di€erent conclusions. Di€erence measures are being used as alternatives but they have not been thoroughly investigated. This paper reviews statistical methods that may be helpful in comparing simulations with measured variables. They have been used to analyse comparisons of simulations produced by a nitrogen response model, N_ABLE, with measurements made in a ®eld experiment with lettuce. The techniques include: (1) data transformation, (2) regression analysis and (3) analysis of di€erence. They show that accuracy of prediction for di€erent variables varied in di€erent growth periods. Mean absolute errors (MAE) were within 0.5% and 35 kg haÿ1 of the measured values of percent-N and soil mineral N, respectively, over the whole growth period. They also demonstrate systematic errors when crop weights exceeded 2 t haÿ1, with 0±0.38 t haÿ1 under-estimation of dry weight and 3±16 kg haÿ1 under-estimation of N uptake when the harvest date was set at values between 42 and 61 days. Use of regression analysis showed that the data sets all violated normality and equal variance assumptions and that choosing the right transformation before analysis was crucially important. When di€erence measures were used, it was found that there was strong correlation between the outcome of some, but not others. Overall, the results suggest that the tests can be grouped according to the degree of correlation between individual tests and that only one test needs to be made from each correlated group. We suggest that two sets of four statistics can be used, each statistic explaining a special property of the data. Each set leads to useful conclusions. The ®rst set is mean of error (E), root mean square error, modi®ed forecasting eciency and paired t-statistic, and the second set is E, MAE, forecasting coecient and F-ratio of lack of ®t over experimental error (FLF ). Either 0308-521X/00/$ see front matter # 2000 Elsevier Science Ltd. All rights reserved. PII : S0308 -521X(00)00010 -X Agricultural Systems 64 (2000) 37±53 www.elsevier.com/locate/agsy * Corresponding author. set can give the same conclusions which could not be quantitatively detected by graphical inspection of the experimental data. # 2000 Elsevier Science Ltd. All rights reserved.